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> ■ ABSTRACT 

(N : 

. We discuss a bias present in the calculation of the global luminosity function (LF) 

' which occurs when analysing faint galaxy samples. This effect exists because of the 

different spectral energy distributions of galaxies, which are in turn quantified by the 
fc-corrections. We demonstrate that this bias occurs because not all galaxy types are 
visible in the same absolute magnitude range at a given redshift and it mainly arises 
at high redshift since it is related to large fc-corrections. We use realistic simulations 
Q ■ with observed LFs to investigate the amplitude of the bias. We also compare our 

Qh' results to the global LFs derived from Hubble Deep Field-North and -South (HDF) 

Q . surveys. We conclude that, as expected, there is no bias in the global LF measured 

' in the absolute magnitude range where all galaxy types are observable. Beyond this 

^ I range the faint-end slope of the global LF can be over/under-estimated depending on 

i the adopted LF estimator. The effect is larger when the reference filter in which the 

' global LF is measured, is far from the rest-frame filter in which galaxies are selected. 

The fact that LF estimators are differently affected by this bias implies that the bias 
is minimal when the different LF estimators give measurements consistent with one 
^ I another at the faint-end. For instance, we show that the estimators are discrepant in 

the same way both in the simulated and HDF LFs. This suggests that the HDF LFs 
are affected by the presently studied bias. The best solution to avoid this bias is to 
derive the global LF in the reference filter closest to the rest-frame selection filter. 
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1 INTRODUCTION 

The luminosity function (LF) is a fundamental and basic 
tool to understand and constrain the history of galaxy for- 
mation and evolution. Moreover, the derived mean luminos- 
ity density at different redshifts allows to derive estimates of 
the cosmic star formation density. In the distant Universe, 
LFs are measured in several redshift bins in order to quan- 
tify the evolution of galaxy populations. In this paper, we 
focus on the reliability of the statistical estimators usually 
used to measure the global LF. We call global L F the sum 
of the LFs per galaxy type (iBinggeli et aljll98^ . Calculat- 
ing LFs is not a trivial task since estimators must account 
for all biases or limits introduced by the observational se- 
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lection effects. Most of the surveys are limited in appar- 
ent magnitud e. This effect i s accounted for in the 1/Vmax 
LF estimator iSchmidtlll968l) . The drawback of the 1 / Vmax 
method is the implicit assumption, in its formulation, of 
a uniform galaxy distribution (i.e. no significant over- or 
under-densities of galaxies) . Nevertheless, because of its sim- 
plicity, t his method is the m ost often used in high-redshift 
surveys. iLvnden-Bell' fm^ developed the C method to 
overcome the assumpti on of a uniform galax y distribution. 
The STY USandaee. Tammann fc Yahij|l979)l and the Step 
Wise Maximum-Likelihood LF estim ators, hereafter SWML, 
jEfstathiou. EUis fc Petersonl llQsd. EEP) are both related 
to maximum-likelihood statistical methods. The C~, STY 
and SWML methods make no assumptions on spatial dis- 
tribution of galaxies, but the information about th e normal- 
ization of the LF is lost. iDavis fc HuchrJ il982ll reviewed 
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various estimators to derive the normalization. In contrast 
to C~ and SWML, the STY method does assume a para- 
metric form to the luminosity distribution. 

All LF estim ators present both advantages and dr aw- 
backs . IWillmeJ il997) and Takcuchi, Yosliikawa & I shiil 
l|2000l) compared several LF estimators using simulated ca- 
talogues. Their mock catalogues did not tackle into detail 
the effects of fc-corrections and of the mix of individual and 
different LF shapes for different morphological types in the 
measurement of the global LF. The Canada-France Red- 
shift Survey CCFRS: iLillv et aljUoOsI) demonstrated that 
the evolution of the LF depends strongly on the studied 
galaxy population. In this paper we add the dependency of 
limiting absolute magnitudes on galaxy type in simulated 
catalogues, and at the same time we introduce an evolution 
of the LF per galaxy population to produce realistic simula- 
tions. These improvements enable us to identify an intrinsic 
bias in the estimators to measure the global LF. 

The different visibility limits for the various galaxy 
types (mainly due to different fc-corrections) affects all flux- 
limited surveys. Hence it can have an impact on statistical 
analyses, in particular the LF estimates. The accepted idea 
is that certain galaxy types sometimes can not be visible in a 
given redshift bin, so that it would obviously underestimate 
the global LF. However even though all galaxy types are 
visible in a given redshift bin, we show using realistic sim- 
ulations that a bias st ill arises in the m easurement of the 
global LF. As noted bv lLillv et alj 1^9^, it occurs because 
different galaxy types are not visible in the same absolute 
magnitude range. In the literature this bias has never been 
quantified. We use real and simulated data to investigate 
the amplitude and the behavior of this bias. In particular 
our analysis is focused on high-redshift data since the k- 
correction values are small at low redshift. 

One possible solution to avoid this bias would be to 
sum the extrapolated LFs per galaxy type to measure the 
global LF. Unfortunately this solution is hazardous in the 
highest redshift bins of a deep survey for two reasons; the 
number of galaxies is often too small to derive LFs per 
galaxy type and not all the LF slopes per galaxy type are 
well constrained, which would imply a dangerous extrap- 
olation. The analysis of the global high-redshift LFs has 
been the framework of most of the previous analyses on 
deep surveys like, for instance, the Subaru Deep Field (SDF; 
[Kashikawa ct al. 2003), the Hubble Deep Fields (HDF; e.g. 
for example iSawicki. Lin fc Ye3[l^ . iTakeuchi et al.ll200Cll. 
i Bolzonella. Pello fc Maccagnill2002D the CFRS jLillv et all 
Il995r) . With the on-going or earlier deep surveys, one needs 
to quantify in details this bias related to fc-correction effects. 

This paper is organized as follows. Section 2 describes 
the origin of the bias linked to the spectral energy distri- 
bution dependency of absolute magnitudes. Section 3 re- 
views briefly the following estimators, 1/Vmax, STY, SWML 
and C^, and the bias linked to each of them. Section 4 
quantifies the impact of the bias on the global LF from 
simulations and from the Hubble Deep Field surveys. Sec- 
tion 5 presents our conclusion. Throughout this paper, we 
adopt an Einstein-de Sitter universe (f2o = 1, f^A = 0) and 
Ho = 100 km s"'^ Mpc"'^, but the results here discussed are 
not dependent on the adopted cosmological model. 
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Figure 1. Example of the observable M—z plane in a flux-limited 
survey for elliptical-type SEDs (open circles) limited by the solid 
lines, and irregular-type SEDs (crosses) limited by the dashed 
lines. Each type of galaxies is observable between its respective 
limits of absolute mag nitudes, M^^^^ and M^"^ The M - z 

^ ' origtit J ami 

planes for different SEDs do not coincide. In this example the 
selection filter is the I band and the absolute magnitudes are 
computed in the U band. 

2 ORIGIN OF THE BIAS 

In any flux-limited survey, galaxy i is observed within a fixed 
apparent magnitude range, mf^jg^j ^ mf ^ n^/amti where 
S designates the band where galaxies are selected. We use 
Ref to designate the reference filter corresponding to the 
wavelength at which the LF is estimated. The observational 
limits imply that galaxy i with spectral energy distribution 
SEDi is observable within a fixed redshift range and a fixed 
absolute magnitude range. 

Obviously, the redshift and absolute magnitude limits 
depend on SEDi (see Fig.0. That is, galaxy SEDi with ab- 
solute magnitude, M^"^^ , is observable in the redshift range: 

zbr^ght{M^"^ ,SED,) s; 2. ^ Z/„,„t ( Aff^^ SED, ) • 

Similary galaxy SEDi at redshift Zi is observable in the ab- 
solute magnitude range: 

The latter limits of absolute magnitudes are defined by, 

Kr%hti^^ SED,) = mf„,;., - DM{z) - KC{z, SED,), 

Mf:l,{z, SEDO = mf„„, - DM{z) ~ KC{z, SED.), 

where DM{z) is the distance modulus and KC is expressed 
as follows: 

KC{z, SEDO = (fc^"^(2) + m^(z) - m'^'^f {z)f^°\ 

where k is the fe-correction, and m is the apparent mag- 
nitude measured from the SED. We note that, for typical 
galaxy k-corrections, M^^^i^^ and M^^f^^ are overall strictly 
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decreasing (i.e. increasing in brightness) as a function of 
redshift. Thus, in a given redshift interval, ziow ^ z < 
Zhigh, it is impossible to observe galaxy SEDi fainter than 
^fa'iL(«io"..SEDO and brighter than M,^^^^,(zMgh, SED,). 
Then, in a given absolute magnitude interval, some SEDs 
may not be observable while others are detected. This leads 
to a bias intrinsic to the global LF estimated in a given red- 
shift interval, ziow ^ z < Zhigh, as we describe in details in 
Section 3. We discuss in this paper the bias in the luminos- 
ity function induced by different SEDs. However a similar 
bias arises when using a magnitude limited sample to de- 
rive estimates of other distribution functions, like masses or 
sizes. 

Of course, no bias would be present if the entire popu- 
lation under study had the same or very similar SEDs, as in 
the case of the estimate of the LF for a single galaxy type. 
When this is not the case, as for example in the case of galax- 
ies of all types (from very blue to very red) an obvious bias 
is arising. As shown in Fig.|51 there are three possible cases, 
which are defined on the basis of z, X'^ (effective wavelength 
of the selection filter) and X^"^^ (effective wavelength of the 
reference filter): 

• 1 + ziow < X^ /X^"-^ , the faint limiting absolute magni- 
tude is brighter for blue galaxies than for red galaxies and 
therefore faint blue galaxies are missing from the sample; 

• 1 + Ziow ~ X^ /X^'^^ , the faint limiting absolute magni- 
tude is about the same for all galaxies; 

• 1-1- Ziow > X^ /X^'^^ , the faint limiting absolute magni- 
tude is brighter for red galaxies than for blue galaxies and 
therefore faint red galaxies are missing from the sample. 

The impact of the bias on each LF estimator depends on 
the case in which the analysis of the global LF is per- 
formed and on the different slopes and normalizations of 
red and blue galaxies. One possible way to cope with this 
effect is to derive the global LF in an absolute magni- 
tud e range in which all types are o bservable as done in 
e.g. lSmall. Sargent fc HamiltonI (^23)- However this means 
to throw away from the analysis some fraction of the data. 
Another way is to define the sample to be analysed in such 
a w ay that 1 -|- ziow ~ X^/X^"^^ for each redshift bin (see 
e.g. IPoh et"alll200ll 2003). This requires the availability of 
a multi-color survey, if one wants to estimate the LF in a 
fixed A^'^-'^ over a wide redshift range. 

Fig. 121 shows the faint observable absolute magnitude 
limits, Mf^/^^, as a function of z for three SEDs (E, Sp, 
Irr) in the case of an /-selection filter, m^faint = 26 mag. In 
Fig. 121 three reference filters are considered: UV(2000 A), 
B(4500 A) and 1(8140 A). For illustration purposes, we con- 
sider galaxy populations in the redshift range 0.70 ^ Zi < 
1.25. Let us consider the case of the UV reference filter (top 
panel of Fig.|2j. Irregular- type SEDs are visible in the whole 
z interval [0.70, 1.25] if they are brighter than M^^ = -16.0 
(case a). They are still visible, but not in the whole z in- 
terval, if -16.0 < M^^ < -14.2 (case b). Finally, irregular- 
type SEDs fainter than M^^ — —14.2 mag are not visible 
at all in this z interval (case c). In case a, no correction 
for missing objects has to be applied when computing the 
LF. In case b, LF estimators enable to correct for the fact 
that some galaxies cannot be visible in the whole redshift 
range. In case c, the global LF cannot be measured because 
irregular- type SEDs are lost. The same considerations apply 
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Figure 2. Faint observable absolute magnitude limits in different 
reference filters as a function of redshift for various SEDs (see Sec- 
tion 2). The filter to select galaxies is / with Iab ^ 26 mag. The 
reference filters are respectively the UV FOCA filter (2000 A), the 
B HST filter (4500 A), the I HST filter (8140 A). In each panel, 
wc show three templates from a set of SEDs described in Sec- 
tion 4.1; solid, dotted and short-dashed lines represent elliptical-, 
spiral- and irregular-type SED galaxies respectively. Vertical long 
dashed lines define the studied redshift interval, 0.7 ^ z < 1.25. In 
the top panel, we illustrate positions of observable galaxies with 
open circles for elliptical- type SEDs, filled triangles for spiral- type 
SEDs, and crosses for irregular-type SEDs. 



for spiral- and elliptical-type SEDs, except that the observ- 
able M'^^ limits have different values. In our first exam- 
ple (UV reference filter) , faint galaxies of the irregular-type 
SEDs become unobservable in an absolute magnitude range 
where galaxies with other SEDs can still be detected. The 
resulting estimate of the global LF would start to be biased, 
because of the absence from the sample of these galaxies, 
for absolute magnitude bins fainter than M^^ = -14.2. As 
shown in the top panel of Fig. |21 in this example the lu- 
minosity function for elliptical type SEDs can be estimated 
down to M^^ ~ —10.0. As a consequence, the range of 
magnitudes over which the estimate of the global LF would 
be biased is of the order of 4 magnitudes. Obviously, this 
range is much smaller and close to zero when the rest frame 
wavelength corresponding to the selection filter is close to 
the reference filter (see, for example, the middle panel in 
Fig. 121 where Ref — B). In case Ref = I, the biased range 
spans about one magnitude (bottom panel of Fig. |21l, but 
in this case are the early type SEDs which are missing from 
the sample in an absolute magnitude range where irregular 
type SEDs are still visible. 

In summary, the bias we are discussing arises when esti- 
mating the LFs of the entire galaxy population in deep sur- 
veys because of the dependency of limiting absolute magni- 
tudes on galaxy types. In estimating LFs, particular care has 
to be taken in the faintest absolute magnitude bins, where 
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some SEDs are no longer observable, thus introducing a bias 
in the measurement of the LPs. There is no bias in a the ab- 
solute magnitude range where all SEDs can be observed. 
In the case where the redshift bin starts at ziow = 0, the 
bias does not appear since all SEDs have the same faintest 
observable absolute magnitude limit. 



3 BIAS INTRINSIC TO THE LUMINOSITY 
FUNCTION ESTIMATORS 

In this section, we describe the intrinsic bias which arises 
when one applies the commonly used LP estimators, 
1/Vmax, C+, SWML and STY to estimate the global LP 
of the population composed by objects with different SEDs. 
We here describe the case of a magnitude limited sam- 
ple with both bright and faint apparent magnitude cuts, 
''^bright m^fainf ^ giveu redshift interval, [ziow, 

Zhigh], the minimum and maximum observable redshifts for 
galaxy i are Zmin,i = max[2;o„, ZbrightiM^"^ ,SEDi)] and 
Zmax,i = ram[zhigh, zfaintiM^"^ ,SE'Di)]. We caU Ng, the 
total number of galaxies observed in the redshift interval, 
Ziow ^ 2 < Zhigh- We first analyse the 1/Vmax and C+ 
estimators which are biased in a similar way with respect 
to the measurement of the global LP; then we analyse the 
maximum-likelihood methods, SWML and STY. 



3.1 The 1/Vmax and C^ estimators 

3.1.1 The l/Vmax estimator 

The maximum observable comoving volume in which 
galaxy i can be detected, is given by 

j2t 



3.1.2 The (7+ estimator 



diudz 



diodz. 



(1) 



where uj is the effective solid angle of the survey, and V is the 
comoving volume. The LP measured in the reference filter 
Ref, (f)^'^^ \M), is discretized in bins of absolute magnitudes 
with width dM as follows: 



^R-fl^M) = (t>k"'WiMt^' - M), 



(2) 



where the window function W is defined as, 

1 if ~dM/2 ^ M^^f - M < dM/2 



otherwise. 



(3) 



and the discrete values of the LP, (j)^"'^ , are derived in each 
absolute magnitude bin k as follows: 



dM = — !— V Ypiiw{M^''f ~ M^'^f) 

Vtotal ^ — ' VobsA 



(4) 



where Vtotai is the comoving volume between ziom and 
Zhigh. Although not necessary, the term Vtotai is introduced 



to point out the weight, 



Vtotai 



applied to each galaxy i. 



Galaxies which belong to the non observable redshift range. 



\Zlo' 



U[2„ 



Zhigh], contribute to through 
this weight. This weighting scheme assumes a homogenous 
galaxy distribution, and it enables the estimator to recover 
the right number of galaxies with the same SEDi in the 
redshift bin. 



iLvnden-Belll il97J) derived the C method. We use a mo- 
dified version of the C", called C+ JZucca et al.lll997^ . The 

contribution of each galaxy i to the LP in the reference filter, 
Ref, can be expressed by the following recursive expression: 



(5) 



where galaxies are sorted from the faintest {j = 1) to the 
brightest absolute magnitude, and C*" is the number of 
galaxies with M'^"^ < M^"^ and zio^ ^ z < Zmax,i. The 
cumulative luminosity function in Eq. |^ is normalized to 
unity at the minimum luminosity of the galaxies in the sam- 
ple. The absolute normalization A is then derived using the 
method described in EEP88. 

The LP in a given absolute magnitude bin k is then the 
sum of the contributions of all galaxies to this bin. 



rRef 



M 



(6) 



^p{M^'^^) is the contribution of galaxy i to (pj^'^^ , and this 
contribution enables to recover the right number of galaxies 
with the same SED^ in the redshift bin. 

3.1.3 Intrinsic bias in the global i/Vmax and LPs 

Let us consider the fc*'' bin of the LP, of width 
dM, centered on AI^"^ . Let us take the following 
example, Mf;l^{zio^, SEDi) = Mf'-^ - dM/2 and 

Mf;;it{zion,,SED2) < M^"'' + dM/2. In this case, both 
SEDs are observed in bin (fc — 1). SEDi and SED2 galaxies 
contribute separately to (j^k-i' ^^'^ thus (l>^^{ is well recove- 
red. In bin k, SEDi galaxies are no more observable and only 
SED2 galaxies contribute to ■ * consequence, (jik"^ 
equal to the LP of SED2 galaxies. In Pig.|21we show the bias 
of the global LP estimated with the 1/Vmax method (open 
circles) and with the C""" method (open squares), adopting 
different input LPs for the two SEDs. In the upper-panel, 
we adopt the same LP for the SEDi late-type and SED2 
early-type galaxies. In the lower-panel, different slopes have 
been used for the input LPs (a = —1.6 for late types and 
a = —0.5 for early types). In both cases, beyond the absolute 
magnitude limit where late-type SEDs are not observable, 
the 1/Vmax and the C"^ methods recover the slope of the 
remaining galaxy population. An other way to explain the 
bias which affects the 1/Vmax and C"*" estimators is the fol- 
lowing. If we derive the individual LPs of SEDi and SED2, 
the estimators recover properly the LPs per type. The abso- 
lute magnitude ranges in which the two LPs are estimated 
are not the same; summing these LPs per type without ex- 
trapolating at fainter absolute magnitudes the LP derived 
for SEDi late- type galaxies is exactly the same thing as 
directly deriving the global LP for the 1/Vmax and C"*" es- 
timators. It clearly appears that in the absolute magnitude 
range [—17.7, —13.5] we are summing the LP of the remain- 
ing type (SED2). In the case of a number of galaxies large 
enough to derive LP per type, one possible solution to over- 
come the bias would be to derive the global LP by summing 
the extrapolated LPs per type. We note that the normal- 
ization (using EEP88 method) is slightly overestimated. We 
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have checked that this normahzation method recovers the 
input normahzation only if the LF is estimated with SWML 
or STY methods, and not with the C*" method. 

In conclusion, the l/Vmax and C"*" methods lead always 
to underestimating the LF in the faintest absolute magni- 
tude bins, thus biasing the global faint-end slope of the LF. 



3.2 The maximum likelihood estimators 

3.2.1 The STY and SWML estimators 

The STY dSandage et alligTST) and the SWML (e.g. EEP88) 
estimators are both derived from maximum-likelihood me- 
thods. The likelihood £ is the probability to obtain a sample 
equal to the observed one within the apparent magnitude 
limits of the survey. £ is computed as the product of the 
probabilities to observe each galaxy at M^"^^ , 

c^ripiMt^f) = ri ^ '^''-^ ) , (7) 

where M^;l^(z„SED,) and Mj^^^^,(z„ SED,) are the faint 
and bright observable absolute magnitude limits of galaxy i 
at Zi (see Section 2). We maximize £ with respect to the 
LF. For the STY estimator, a functional f orm for the LF i s 
assumed. We use the Schechter function iSchechteJlioT^ . 
For the SWML estimator, the LF is discretized into abso- 
lute magnitude bins (see Eq. No assumption is made 
about the LF shape. The STY parametric and SWML non- 
parametric estimators are complementary. The error bars in 
the SWML estimator are derived from the covariance ma- 
trix and the normalization is the sum of the inverse of the 
selection function of each galaxy as described in EEP88. 



3.2.2 Intrinsic bias in the global SWML and STY LFs 

The SWML and STY estimators are biased in a different 
way with respect to the 1/Vmax and C*" estimators. Let us 
write £ as the product of £i and £2, which are respec- 
tively the likelihood for SEDi and SED2 galaxy popula- 
tions within a given redshift interval where their faintest 
observable absolute magnitudes are M^'^^J (ziom, SEDi) < 
-^/I'ilt (^'o«" SED2). The global LF is weU recovered in 
the absolute magnitude range ] — 00, M^^l^^{ziow,^^^i)\ 
where both SEDs are observable. For Mf^f^^{zio^,SKDi) < 

Mf"^ < M^^f„,(2,o„,SED2), the shape of the LF is only 
constrained by the probability of SED2 population, since 
SEDi population is not detected in this range of absolute 
magnitudes. Thus the global LF has the same shape as 
the SED2 LF. In the range ]Mf^/^^{zion,,SEDi),+oo[, the 
SWML and STY methods estimate the shape of the SED2 
LF rather than the shape of the global LF. Three cases may 
occur as follows. First, the LF of SEDi and SED2 popu- 
lations have the same shape; the global LF is well recov- 
ered. Second, the SED2 LF has a flatter faint-end slope; 
the global LF is underestimated. Third, the SED2 LF has 
a steeper faint-end slope than SEDi; the global LF is over- 
estimated. The first two cases are illustrated in Fig. |3 In 
the bottom panel of Fig. |21 the estimate of the LF derived 
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Figure 3. Wc simulate a simple mock catalogue with two SEDs 
of galaxies. SEDi late-type galaxy population has a faint observ- 
able absolute magnitude limit at —17.7 mag (long-dashed verti- 
cal line) and an input LF shown by the long-dashed curved line. 
SED2 early-type galaxy population has a fainter observable limit 
at —13.5 mag (short-dashed vertical line) and an input LF shown 
by the short-dashed curved line. The input global simulated LF 
is the dotted curved line, which is the sum of the input LFs of 
SEDi and SED2. In the top panel, the input LFs for SEDi and 
SED2 are the same. The 1/Vmax LF estimate (circles) and the 
C+ LF estimate (squares) do not recover the input global LF 
at magnitudes fainter than —17.7 mag where SEDi is no more 
observable, and thus the estimate is equal to the SED2 LF. The 
SWML LF estimate (triangles) and the STY estimate (solid line) 
recover the input global LF since their shapes are constrained by 
the shape of SED2 input LF only, which is the same as SEDi in- 
put LF. In the bottom panel, the input LFs for SEDi and SED2 
have a faint-end slope a = —1.6 and a = —0.5 respectively. At 
magnitudes fainter than —17.7 mag, where SEDi population is 
no more detected, the 1/Vmax LF and the C"*" LF are equal to 
SED2 input LF. The SWML and STY LF faint-ends (constrained 
by the shape of SED2 input LF) are underestimated. 

with the SWML method does not follow a Schechter func- 
tion. Thus, the STY estimate, which assumes a Schechter 
parametric function and which has the same behavior as a 
SWML method, is not able to recover the input. 

In conclusion, the bias in the faint-end slope of the 
global LF estimated with likelihood methods depends on 
the LF shape of each SED. Thus to quantify the bias, we 
need to use simulations which deal with LF per SED, as we 
do below. 



4 APPLICATIONS TO SIMULATED AND 
REAL DATA 

In this section, we give a description of the Hubble Deep 
Field (HDF) data and simulations (Section 4.1), then we 
illustrate the bias in a qualitative way using the HDF data 
(Section 4.2), and finally we quantify the bias with 1,000 
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Spectral class 


<!>* X IQ-^Mpc-^ 


a 


Mr 


Early 


14.77 


0.11 


-20.56 


Spiral 


13.61 


-0.73 


-20.43 


Irr 


6.52 


-1.64 


-19.84 



Table 1. Input LF parameters per spectral class in the R band 
(in Vega system) used for the simulations in Section 4. 

simulations of the HDF survey and the Virmos-VLT Deep 
Survey (VVDS) (Section 4.3). 

4.1 Brief description of the data 

We use the public version of t he multi-color 
mock catalogues from lArnout j i2004l) available at 
www. lam. oamp.fr/arnouts/LE_PHARE.html. Simulations 
are based on an empirical approach using observed LPs to 
derive redshift and apparent magnitud e distributions. The 
LPs of the ESO-Sculptor Survey (ESS; Ide Lapparent et all 
|2003^ are implemented up to redshift z < 0.6 for early, 
spiral and irregular spectral types (see Tab. 0. The LP 
evolution per type beyond z ~ 0.6 is constrained in a way 
that it reproduces the observed redshift, number count and 
color distributions. We made the following assumptions 
as in lArnouts ( 200^ : the LPs of early and spiral class 
are constant with redshift; the faint-end slope for the 
three LPs is constant; for the irregular class there is an 
evolution in density in the redshift range 0.15 < z ^ 0.7 
using 4'*{z) = + 3.69(2 — 0.15)), and an evolution 

in luminosity in the redshift range 1.25 < z ^ 2.5 using 
M*{z) = M* - 0.12(z - 1.25). Mock catalogues are derive d 
using the set of SEDs as described in lArnouts et al.l il999t) . 
composed of 72 SEDs t hat have been interpo lated between 
four observed spectra of lColeman et al 7 f 1980) and two star- 
burst models, computed with GISSEL tiBruzual fc Charlod 
Il993l) . The set of SEDs is divided into three main spectral 
classes: elliptical, spiral and irregular galaxies. Since a 
spectral class corresponds to several different SEDs, the 
bias may arise also within a single class. To test this last 
point, we also use only one SED per spectral class, even 
though the simulation with a set of only three SEDs is less 
realistic. 

We used the photometric catalogue and the photomet- 
ric redshifts o f the HDF South and North surveys from 
lArnouts et all (1999. l20o3) . Our aim is to illustrate the bias 
with observational data, and it is not to derive the best LP 
of the HDF. The HDF survey is taken as an example of a 
survey in which, because of the relatively small number of 
observed galaxies, only the global LF, rather than the LP 
for different galaxy types, can be reliably derived in different 
redshift bins. For consistency in the comparison, the HDF 
data have been analysed using exactly the same set of 72 
SEDs as in the simulation. 

4.2 Qualitative description of the bias 

We describe the bias using HDF data and one simulation 
from which data have been selected with the same apparent 
magnitude limits as the HDP data. To ilUustrate the bias 
in a qualitative way, we use a large number of galaxies 



(> 5,000 in each redshift bin) so that statistical uncertain- 
ties become negligible. Moreover we did not include any 
surface brightness effect and any uncertainties on apparent 
magnitudes, redshifts, and absolute magnitudes. We derived 
the absolute magnitudes and the global LPs in two redshift 
intervals: 0.70 ^ z < 1.25 and 1.25 4, z < 2.00 in the follow- 
ing reference filters, UV (Fig.^Ji), B (Pig-Et)) and I (Fig.|IJ:) 
with samples selected at Iab ^ 26 mag. In these figures, we 
plot the input LPs of the simulation (one for each spectral 
type and the global one) and the estimates of the global LP 
derived by the 1/Vmax, C+, SWML and STY methods from 
mock catalogues and the HDF data. Before discussing in 
detail each figure, we note that in all cases the estimators 
give similar results using the mock catalogues constructed 
with three SEDs only and those with 72 SEDs. We display 
the simulations with three SEDs so to compare exactly to 
the three input simulated LPs per class (i.e. per SED in 
this case). With 72 SEDs such a comparison is less obvious 
since each class contains several SEDs. However we show 
also the simulations with 72 SEDs so it is comparable to 
the analysis applied to the HDF data, that is with the same 
selection function, the same set of SEDs and the same filters. 



4.2.1 UV-rest LFs 

In the left panels of Pig. we show the individual input 
LPs (dotted lines) of the mock catalogue built with three 
SEDs, that is with one SED per spectral class. The sum 
of the three individual input LPs corresponds to the global 
simulated LF (solid line). Thus the LF estimators should 
be able to recover this global LP in estimating the LP for 
the whole galaxy sample. First, we note that all estimators 
underestimate the faint-end of the LPs. As previously shown 
in Fig. |5| the irregular-type galaxies 'disappear' from the 
faintest absolute magnitude bins (for M^^ > —14.2 mag 
at ziow = 0.7 and M^^ > —16.0 mag at ziow = 1.25). As 
explained in Section 3 when one type 'disappears' the global 
LP estimate is based only on the remaining types. 

Left panels of Fig. show this point; the 1/Vmax and 

estimators recover the input LF of elliptical galaxies in 
the faintest absolute magnitude bins (i.e. open circles and 
squares overlap the dotted line of the elliptical- type LF). 
The SWML method underestimates the faint-end slope of 
the global LF since this estimator recovers the shape of the 
elliptical-type galaxies (i.e. open triangles follow the same 
slope as the elliptical- type LF). The less biased result is ob- 
tained with the STY method and the more biased results are 
obtained with the 1/Vmax and C"*" methods. It is interesting 
to note in the redshift interval 1.25 ^ z < 2.0 that the little 
'bump' of the global SWML LP estimate at M"^ ~ -15 
mag corresponds to the shape the elliptical-type input LF. 

In the middle panels, we show the global LP derived us- 
ing the mock catalogue with 72 SEDs distributed into three 
spectral classes. We see that the increase of the number of 
SEDs used per spectral class does not affect significantly 
the trend of the LF estimators described above. Using many 
SEDs smooths the effect of the 'disappearance' of a SED. 

In the right panels, we show the global LF derived from 
the HDP data using the same 72 SEDs as in the mock cata- 
logue. We do not observe very faint (and very bright) galax- 
ies as observed in the simulation even though both data sets 
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Figure 4. a Global luminosity functions derived in the UV-FOCA (2000 A) reference filter in two redshift intervals, 0.70 ^ z < 1.25 (top 
panels), 1.25 ^ z < 2.0 (bottom panels). The left and middle panels correspond to the LFs derived from simulations with 3 and 72 SEDs 
respectively. The right panels correspond to the LFs derived from the HDF-North and -South surveys. The limiting magnitude in all cases 
is /as(8140 A) < 26 mag. In the left panels, we display the LFs corresponding to the three input SEDs used (dotted lines): from the 
steepest to the shallowest slope, it is the irregular-, spiral- and elliptical- type LF respectively. In each panel, we plot the global simulated 
LF (solid line) corresponding to the sum of the three input LFs. We plot also the results from the following global LF estimates, STY 
(dashed line), SWML (triangles), C"*" (squares), and 1/Vmax (circles). Below the z redshift intervals we quote the number of galaxies 
used to derive the LFs. We adopt Poissonian error bars for the 1/Vmax and C+ estimators. The error bars for the SWML estimator are 
derived following the EEP88 method. 



have the same selection function. We have tested that this 
is indeed due to the relatively small number of galaxies ob- 
served in the HDF and the small probability to observe the 
faintest (and brightest) gala^cies. Thus, the difference be- 
tween the various estimators is less obvious than in the sim- 
ulation. However we note here the same general trend in the 
faintest observed bins. This may suggest that the bias which 
affects the global HDF LF estimation is similar to the bias 
shown in the simulation. In conclusion the faint-end of the 
global LF cannot be properly recovered by any estimator as 



can be seen from the comparison with the global simulated 
LF (solid line). 



4.2.2 B-rest LFs 

We plot in Fig^Jj the global LF in the B reference filter with 
the same symbols as in Fig^J.- The simulations built with 
both three and 72 SEDs give exactly the same result. At z = 
0.7, the B-filter absolute magnitudes are almost independent 
on SEDs since the selection filter (J) roughly corresponds 
to the B-rest filter at this redshift. As a consequence, all 
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Figure 4.b Same as Fig. except that the reference filter is B-HST (4500 



estimators recover very well the global simulated LF in the 
first redshift bin (0.70 ^ z < 1.25). For the HDF data, the 
four estimators are in agreement amongst them. The bias is 
not present in this shown by our simulation. 

In the redshift bin 1.25 ^ z < 2, the SWML and the 
STY methods slightly overestimate the slope of the global 
LF. At ziow = 1.25, the limiting absolute magnitude M^^^^^ 
for early types is about one magnitude brighter than for late 
types. Thus the SWML and STY estimators measure the 
steep slope of the late-type galaxies, and this explains the 
overestimate of the faint-end slope of the global measured 
LF. The 1/Vmax and methods well recover the global 
simulated LF, since the density of the first type which 'dis- 
appears' from the sample (that is the faint elliptical-type 
galaxies) is negligible in the global simulated LF. Global LF 
estimators from the HDF data are globally in agreement 
amongst them. The estimate of the global LF in the B ref- 
erence filter is quite robust. 



4.2.3 I -rest LFs 

For the /-rest LFs, the range of absolute magnitudes in 
which difi'erent spectral types 'disappear', increases with 
redshift since the reference filter is also the filter used to se- 
lect the sample. At z = 0.7, this range spans one magnitude 
(see Fig. I^J, thus all estimators should be slightly biased in 
the redshift bin 0.7 ^ z < 1.25. We note that all LF estima- 
tors recover the global simulated LF in simulations derived 
with three as weU as with 72 SEDs. For the HDF LF, the 
estimators are in good agreement with each other and this 
suggests that the measurement is not biased. 

At z = 1.25, the absolute magnitude range in which dif- 
ferent spectral types 'disappear' spans more than two magni- 
tudes. In this case the first spectral types which 'disappear' 
from the faintest bins are the elliptical types. In both simu- 
lations, the SWML and STY methods largely overestimate 
the slope of the faint-end in the redshift bin 1.25 ^ 2 < 2.00, 
while the 1/Vmax and C"*" methods recover reasonably well 
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Figure 4.c Same as Fig. except that the reference filter is I-HST (8140 



(the 1/Vmax method better than the method) the faint- 
end of the global LF. Indeed, the slope estimated with the 
SWML and STY methods is the slope of the irregular galax- 
ies, which is steeper than the global input LF slope in the 
range of absolute magnitudes here measured. The 1/Vmax 
and C"*" estimators recover the slope of the global LF since 
the contribution of the faint elliptical-type galaxies to the 
global LF is negligible. In the global LF derived from the 
HDF data, the SWML and STY estimators predict a steeper 
slope than the 1/Vmax and C"*" estimators, as in the sim- 
ulations. According to the simulations, the global LF de- 
rived from the HDF data and estimated with the SWML 
and STY overestimate the faint-end slope in the redshift 
bin 1.25 ^ z < 2.00. 



4.3 Quantitative measurement of the bias 

As shown throughout the paper, the bias depends on the 
band in which galaxies are selected, on the apparent lim- 



its of the survey, on the reference band in which the LF is 
measured, and on the lower limit of a studied redshift bin. 
Thus we cannot provide here a recipe to quantify the am- 
plitude of the bias in all cases. We choose to quantify it in 
the following specific cases, the HDF and the Virmos-VLT 
Deep Survey (VVDS) surveys. The HDF is a good example 
where we can derive the global LF only. The VVDS is a deep 
spectrosc opic survey with enough galaxies to derive the LFs 
per type ihe Fevrell200i) . The global LF is also a necessary 
output of the VVDS survey, however even in this case, de- 
riving it from the sum of extrapolated LFs per type may be 
dangerous. We produced 1,000 simulations representative of 
HDF and VVDS surveys. Realistic apparent magnitude er- 
rors are introduced in each simulation, and this has resultant 
uncertainties on the template fit, k-corrections and absolute 
magnitudes. We do not include any surface brighness effects 
and errors on the redshift. 

We choose the Schechter parameters to quantify the 
bias. The parameter a is the most affected by the bias. In 
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consequence, we fix M* for the LF estimation of each simu- 
lations. We call Aa, the difference between the estimated a 
value and the input a value to the simulations and Aa the 
average value of Aa over 1,000 realizations. 

In the simulations, we have implemented LFs per type. 
Then to compare the estimates of the global LF with the 
input, we had to define a 'pseudo' input global LF. We did 
that in the following way. 

• We realize one big simulation with more than 1,000 
galaxies in each redshift bin so to minimize statistical fiuc- 
tuations. 

• We do not include any errors on redshifts, apparent 
magnitudes, absolute magnitudes, and types. 

• To obtain a global LF which is not affected by the bias 
in the big simulation, we select galaxy samples three mag- 
nitudes fainter than the apparent magnitude limits of the 
1,000 simulations representative of HDF and VVDS. And 
then the absolute magnitude range is limited to the same 
absolute magnitude range spanned by the LF estimates of 
these 1,000 realizations. The Schechter parameters are de- 
rived with the STY method. 

4-3.1 Bias quantified for HDF 

Each simulation of the HDF is realized on 8 arcmin^ and 
with Iab ^ 26. The LFs are derived in three reference bands 
(UV-FOCA, B-HST and I-HST) and in four redshift bins 
from z — 0.5 to z — 2. The results of these simulations are 
shown in Fig. |S] 

In the UV (2000 A) band (first column of the figure), the 
estimate of the faint-end slope with the 1 / Vmax fit is strongly 
underestimated (Aa > 0.5 for z ^ 0.75), while the STY 
estimate is only slightly biased (Aa ~ 0.1). In the B-HST 
band (second column), the LF estimates derived with the 
STY and the 1/Vmax methods are robust up to the redshift 
bin [1, 1.5]. In the redshift bin [1.5, 2], the STY estimate 
gives ^ —0.30 steeper values for the a parameter, whereas 
the 1/Vmax fit gives ~ 0.20 fiatter values. In the I band 
(third column) , the bias from the STY estimate increases as 
a function of redshift; the input a parameter is recovered in 
the first redshift bin, and Aa decreases down to —0.40 in the 
last redshift bin [1.5, 2]. The 1/Vmax fit gives an estimate 
which recovers the input a up to the reshift bin [1, 1.5]. 

4.3.2 Bias quantified for VVDS 

We quantify now the amplitude of the bias for a sample se- 
lected in a way similar to the VVDS. We adopt the Johnson 
Kron-Cousins filter set and we perform each simulation on 
360 arcmin^. The sample selected from each simulation is 
approximatively of the same size (~ 7, 000 galaxies) as the 
presently available spectroscopic deep sample in the 0226- 
0430 VVDS field. The global LF estimations are derived 
in three reference bands, U (3600 A), B (4200 A) and I 
(8000 A), and in four redshift bins from z — 0.6 to 2: = 1.5. 
For our illustration, we consider two cases of sample selec- 
tion; an I-selected sample (as the VVDS spectroscopic data), 
and an U-selected sample (as for instance, a VVDS photo- 
metric data). 

The results of these simulations are shown in the Fig.|S| 
We show in the first column the U-band LF estimates for 



samples selected in the U band with Uab ^ 24. The bias 
increases with redshift. In all redshift bins, the STY estimate 
gives steeper faint-end slopes (Aa ~ —0.3), and the 1/Vmax 
fit gives fiatter faint-end slopes (Aa > 1 for z ^ 1). 

In the second column, samples are selected with Iab ^ 
24, and the LF estimates are derived in the U band. Like 
for the HDF, the faint-end slope is underestimated by both 
the STY method and the 1/Vmax fit in the first redshift bin 
(Aa ~ 0.2 and Aa ~ 0.5 respectively). The bias decreases 
with redshift. In the last redshift bin, all estimates recover 
the input a value because the rest-frame selection I-band 
corresponds to the reference U-band at z ~ 1.2. 

In the third column, samples are selected with Iab ^ 
24, and the LF estimates are derived in the reference B- 
band. The input a parameter is recovered by both methods 
up to z = 1. The STY estimation of a in the redshift bin [1, 
1.2] is only slightly biased (Aa ~ —0.1), while it is strongly 
biased in the last redshift bin [1.2, 1.5]. The STY estimate 
gives steeper faint-end slopes (Aa ~ —0.4), and the 1/Vmax 
fit gives flatter faint-end slopes (Aa ~ 0.2). 

In the last column, samples are selected with Iab ^ 24, 
and the LF estimates are derived in the reference I-band. 
The bias increases with redshift. The 1/Vmax fit gives 
fiatter faint-end slopes (up to Sa ~ 0.4 for 1.2 ^ z < 1.5). 
The STY estimate gives steeper faint-end slopes (up to 
Aa ~ -1 for 1.2^ z< 1.5). 

In conclusion, in deep surveys like the HDF or the 
VVDS, the various estimators fail to correctly recover the 
global LF faint end if the reference filter is far from the rest- 
frame selection filter. If the results from different estimators 
are not in good agreement with each other, the global LF 
estimate is likely to be biased and the only way to quanti- 
tatively estimate this bias is through a detailed comparison 
with simulations representing to the sample properties. 



5 CONCLUSIONS 

Our study enabled us to describe when LF estimators are ro- 
bust for the measurement of the global LF in the framework 
of the earlier and future deepest surveys. We demonstrated 
that the estimation of the global LF contains an intrinsic 
bias due to the fact that, in a magnitude limited sample, 
different galaxy types have different limits in absolute mag- 
nitude because of different k-corrections. The importance of 
the effect is larger when the range of k-correction between 
the different galaxy types is wide. For this reason this bias 
mainly arises in high redshift samples. The STY and SWML 
estimators are not affected in the same way by this bias as 
the 1/Vmax and C+ estimators. If the STY, SWML and 
the 1/Vmax, C"*" methods are not in good agreement with 
each other, this is an indication that the bias in the global 
LF estimators is present. A good indication of the presence 
of a significant bias is when the differences between differ- 
ent estimators (Vmax and STY for instance) is larger than 
the statistical uncertainties (Poisson errors for instance). We 
quantified it using realistic simulations and observations for 
galaxies selected in the 7 filter, and measuring the LF in 
various reference filters (UV, B, I). We obtain the following 
results. 

(i) Case 1 -I- ziow < /X^"^-^ (e.g., a reference-frame UV 
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Figure 5 . Histograms of the differences, Aa, between the estimated and the input a parameters, over 1,000 realizations done with the 
HDF characteristics. Aa for the STY estimates is the solid line histogram, while Aa for the 1/Vmax fit is the dashed-line histogram. All 
the simulated HDF samples are selected with Iab ^ 26. Panels in the first column correspond to the case with the global LPs derived in 
the UV-FOCA (2000 A) filter, in the middle column in the B-HST (4500 A) filter, and in the right column in the I-HST (8140 A) filter. 
From top to bottom panels, Aa is measured within the redshift bins [0.5, 0.75], [0.75, 1], [1, 1.5], [1.5, 2]. 



LF for galaxies selected in I): the studied estimators under- 
estimate the faint-end slope of the global LF for ziow ^ 2. 
This underestimate is particulary significant for the 1 / Vmax 
and C"*" methods (i.e. for instance, the UV-LF of the SDF). 
(ii) Case 1 -I- ziow ~ /X^"^^ (e.g., a reference-frame B LF 
for galaxies selected in I): the estimators of the global LF 
are robust up to ziow In this redshift range the bias 

is minimal (i.e. for instance, the CFRS case), 
(iii) Case 1 -I- ziow > X^ /X^'^^ (e.g., a reference-frame I LFs 
with galaxies selected in I): the STY and SWML methods 
overestimate the faint-end slope of the global LF, while the 
1/Vmax method roughly recovers well the global LF (e.g., 
for instance, the redshift bin [1.25, 2] of the HDF). 

The ways to reduce the intrinsic bias of the global LF 
estimators are the following: 



(a) The selection of galaxy subsamples in the closest rest- 
frame filter to the reference fil ter in which the LF is mea- 
sured (see e.g. iPoh et al.ir200ll 2003). This method is also 
the best to reduce the SED dependency in the measurement 
of absolute magnitudes since in this case the term [color-|-fc- 
correction] is little dependent on the SED. Only multi-color 
surveys allow to derive the same rest-frame band LF at dif- 
ferent redshifts using this strategy. 

(b) In principle, the estimate of the global LF using the sum 
of the extrapolated LF per galaxy type. However it requires 
a good knowledge of the slope for all the LFs per type, and 
in practice the use of extrapolated LFs may be hazardous. 

(c) The estimate of the global LF using a filter in which the 
differences between fc-corrections are small, as for instance 
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Figure 6. Same as in Fig.lSlexcept that the simulations are selected in a way similar to the VVDS. Panels in the first column correspond 
to samples selected with Vab ^ 24, and the other columns correspond to samples selected with Iab ^ 24. In the first and second 
column, the global LFs are derived in the U (3600 A) reference filter, the third column in the B (4300 A), and the last column in the 
I (8000 A). From top to bottom panels, Aa is measured within the redshift bins [0.6, 0.8], [0.8, 1], [1, 1.2], [1.2, 1.5]. 



m the iC-filter, e.g. 

zonella et al.l lj2QQ2)i IPozzetti et alJ 

J2003D . 

(d) The estimate of the global LF within an absolute mag- 
nitude range in w hich all galaxy types are detected (see e.g. 
ISmall et alll99'^ . This method is appropriate for very large 
surveys like the VVDS for instance, at the cost of the loss 
of the faintest bins of the global LF. 
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